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Detailed Action 

1 . This office action is in response to an AMENDMENT entered December 19, 2007 
for the patent application 10/773050 filed on February 5, 2004. 

2. All previous Office Actions are fully incorporated into this Non-Final Office Action 
by reference. 



Status of Claims 

3. Claims 43-48, 51 -67, 70-88 are pending. 



Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 
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Claims 43-48, 51-57, 59-67, 70-76, 78-80 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Oh in view of Lehmann. ('Analog CMOS Implementation of 
Artificial Neural Networks for Temporal Signal Learning', referred to as Oh; 'Mixed 
analog/digital matrix-vector multiplier for neural networks synapses', referred to as 
Lehmann) 

Claim 43 

Oh teaches an array of synaptic cells, which are interconnected to form a 
feedforward neural network, wherein each synaptic cell includes. (Oh, p4:1 9 through 
P 5:2) 

Oh does not teach a learning electrical circuit operable to update a local weight 
according to an update rule and store the local weight in a capacitor in the cell, a digital 
memory operable to store the local weight in a digital form. 

Lehmann teaches a learning electrical circuit operable to update a local weight 
according to an update rule and store the local weight in a capacitor in the cell, a digital 
memory operable to store the local weight in a digital form. (Lehmann, abstract, p61 , 
C1 :36 through C2:21 : 'Learning electrical circuit operable to update a local weight 
according to an update rule' of applicant is equivalent to 'analog storage (a capacitor) 
which would accumulate weight changes during learning' of Lehmann. 'A digital memory 
operable to store the local weight in a digital form' of applicant is equivalent to digital 
weight' of Lehmann.) It would have been obvious to a person having ordinary skill in the 
art at the time of applicant's invention to modify the teachings of Oh by using both 
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analog and digital storage devices as taught by Lehmann to have a learning electrical 
circuit operable to update a local weight according to an update rule and store the local 
weight in a capacitor in the cell, a digital memory operable to store the local weight in a 
digital form. 

For the purpose of having low hardware overhead for learning. 

Oh teaches a processing electrical circuit comprised of a one-dimensional 
multiplier which is operable to process a component of an analog input signal in 
accordance with the local weight stored in the digital memory. (Oh, abstract; 'One 
dimensional multiplier' of applicant is equivalent to 'vector multiplier' of Oh.) 

Claim 44 

Oh teaches wherein each column of synaptic cells in the array of synaptic cells 
represents a neuron in the neural network. (Oh, Figure 6.2; One function of a neuron is 
to use its output for training to convergence. The column(s) of Figure 6.2 illustrate this 
by using the outcome of 'error' circuit for training purposes.) 

Claim 45 

Oh teaches an array of control cells positioned adjacent to the array of synaptic 
cells, where each control cell controls conversion of the local weight from an analog 
form to a digital form for a row of synaptic cells in the array of synaptic cells. (Oh, 
p1 16:2-1 7; Oh discloses an analog to digital converter to have analog input stored in 
digital form.) 
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Claim 46 

Oh teaches wherein the control cells operate synchronously with each other to 
control conversion. (Oh, p12:14 through p13:3; Oh discloses in the McCulloch-Pitts 
model, in which neurons operate in asynchronous manner. Using Oh's model in which 
the feedforward neural network is interconnected allows synchronously operations to 
occur.) 

Claim 47 

Oh teaches wherein each control cell includes a multiplexer configured to select 
a synaptic cell in a given row of synaptic cells (Oh, figure 6.2; Figure 6.2 illustrates 
numerous multiplexers within the recurrent neural network.) and an analog-to-digital 
converter. (Oh, p1 16:2-17) 

Claim 48 

Oh teaches wherein the analog-to-digital converter for a given row of synaptic 
cells is interconnected to each synaptic cell in the given row of synaptic cells. (Oh, 
p1 16:2-17) 

Claim 51 

Oh teaches a switch interposed between the processing circuit and the digital 
memory for selectively enabling use of the digital memory. (Oh, p1 1 6:2-1 7; Oh 
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discloses a second recurrent neural network (slave neural network) which employs the 
weight values from the first recurrent neural network. These values are updated using 
the digital memory. If a user can alter between the first and second recurrent neural 
network, then there exists a 'switch' to do so.) 

Claim 52 

Oh teaches a switch interposed between the processing circuit and the learning 
circuit for selectively enabling use of the learning circuit. (Oh, p1 1 6:2-1 7; Oh discloses a 
first recurrent neural network (master neural network) which employs the learning phase 
of the neural network or learning circuit of applicant. These values are updated using 
the digital memory. If a user can alter between the first and second recurrent neural 
network, then there exists a 'switch' to do so.) 

Claim 53 

Oh teaches a switch interposed between the processing circuit, the learning 
circuit and the digital memory for selectively enabling the use of either the learning 
circuit or the digital memory by the processing circuit. (Oh, p1 16, 2-17; If there exists 
two phases, a learning phase and a testing phase, then there exists a switch to alter 
between the two phases.) 



Claim 54 
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Oh teaches the learning circuit is comprised of a one-dimensional multiplier for 
implementing the update rule. (Oh, abstract; 'One dimensional multiplier' of applicant is 
equivalent to 'vector multiplier' of Oh.) 

Claim 55 

Oh teaches the learning circuit is configured to receive an error signal indicative 
of a difference between an output signal and a target output signal. (Oh, p15:9 through 
p16:7; 'Output signal' and 'target output signal' of applicant is equivalent to 'actual 
output' and 'desired output' of Oh.) 

Claim 56 

Oh teaches multiple arrays of synaptic cells, which are interconnected to form a 
multi-layer neural network. (Oh, figure 6.2; Figure 6.2; The 2 dimensional array which 
each node is interconnected with other nodes of Oh is equivalent to 'multi-layer neural 
network' of applicant.) 

Claim 57 

Oh teaches the learning circuit further comprises a second multiplier configured 
to receive the error signal and generates a feedback propagation error signal for a 
previous layer in the multi-layer neural network.(Oh, p1 7:5-6 and equation (2.1 1); 'Error 
signal' of applicant is equivalent to 'w k ' of Oh. 'Previous layer' of applicant is equivalent 
to 'w k " 1 'ofOh.) 
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Claim 59 

Oh teaches a digital-to-analog converter interposed between the processing 
circuit and the digital memory in each synaptic cell. (Oh, p1 1 6:2-1 7; Oh discloses a 
digital to analog converter for outputting the value of the synaptic cell.) 

Claim 60 

Oh teaches wherein each processing circuit in a column of synaptic cells outputs 
a component of a weighted sum. (Oh, p13:4 through p15:8; 'Processing circuit in a 
column' of applicant is equivalent to 'the 'hidden layer' of Oh. 'Outputs a component of 
a weighted sum' of applicant is equivalent to the output of each node in the hidden layer 
is a result of the weighted sum input.) 

Claim 61 

Oh teaches wherein an output of each multiplier in a column of synaptic cells is 
interconnected to form a weighted sum. (Oh, figure 5.5; Oh discloses the output of each 
multiplier (Y1-Y4) is fed back as a column to form an interconnection of synaptic cells.) 

Claim 62 

Oh teaches an array of synaptic cells which are interconnected to form a 
feedforward neural network (Oh, p4:19 through p5:2) and configured to receive an 



Application/Control Number: 10/773,050 Page 9 

Art Unit: 2129 

analog input signal indicative of a biological cell measurement and to model a process 
of the biological cell, wherein each synaptic cell includes (Oh, p3:6-10; 'Configured to 
receive ... analog input' of applicant is disclosed by a 'subthreshold analog CMOS VLSI' 
of Oh.) 

Oh does not teach a learning electrical circuit operable to update a local weight 
according to an update rule and store the local weight in a capacitor in the cell, a digital 
memory in the cell operable to store the local weight in a digital form. 

Lehmann teaches a learning electrical circuit operable to update a local weight 
according to an update rule and store the local weight in a capacitor in the cell, a digital 
memory in the cell operable to store the local weight in a digital form. (Lehmann, 
abstract, p61 , C1 :36 through C2:21 : 'Learning electrical circuit operable to update a 
local weight according to an update rule' of applicant is equivalent to 'analog storage (a 
capacitor) which would accumulate weight changes during learning' of Lehmann. 'A 
digital memory operable to store the local weight in a digital form' of applicant is 
equivalent to digital weight' of Lehmann.) It would have been obvious to a person having 
ordinary skill in the art at the time of applicant's invention to modify the teachings of Oh 
by using both digital and analog storage devices as taught by Lehman to have a 
learning electrical circuit operable to update a local weight according to an update rule 
and store the local weight in a capacitor in the cell, a digital memory in the cell operable 
to store the local weight in a digital form. 

For the purpose of having low hardware overhead for learning. 
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Oh teaches a processing electrical circuit comprised of a one-dimensional 
multiplier which is operable to process a component of the analog input signal in 
accordance with the local weight stored in the digital memory. (Oh, abstract; 'One 
dimensional multiplier' of applicant is equivalent to 'vector multiplier' of Oh.) 

Claim 63 

Oh teaches wherein each column of synaptic cells in the array of synaptic cells 
represents a neuron in the neural network. (Oh, Figure 6.2; One function of a neuron is 
to use its output for training to convergence. The column(s) of Figure 6.2 illustrate this 
by using the outcome of 'error' circuit for training purposes.) 

Claim 64 

Oh teaches an array of control cells positioned adjacent to the array of synaptic 
cells, where each control cell controls conversion of the local weight from an analog 
form to a digital form for a row of synaptic cells in the array of synaptic cells. (Oh, 
p1 16:2-17; Oh discloses an analog to digital converter to have analog input stored in 
digital form.) 

Claim 65 

Oh teaches wherein the control cells operate synchronously with each other to 
control conversion. (Oh, p12:14 through p13:3; Oh discloses in the McCulloch-Pitts 
model, in which neurons operate in asynchronous manner. Using Oh's model in which 
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the feedforward neural network is interconnected allows synchronously operations to 
occur.) 

Claim 66 

Oh teaches wherein each control cell includes a multiplexer configured to select 
a synaptic cell in a given row of synaptic cells (Oh, figure 6.2; Figure 6.2 illustrates 
numerous multiplexers within the recurrent neural network.) and an analog-to-digital 
converter. (Oh, p1 16:2-17) 

Claim 67 

Oh teaches wherein the analog-to-digital converter for a given row of synaptic 
cells is interconnected to each synaptic cell in the given row of synaptic cells. (Oh, 
p116:2-17) 

Claim 70 

Oh teaches a switch interposed between the processing circuit and the digital 
memory for selectively enabling use of the digital memory. (Oh, p1 1 6:2-1 7; Oh 
discloses a second recurrent neural network (slave neural network) which employs the 
weight values from the first recurrent neural network. These values are updated using 
the digital memory. If a user can alter between the first and second recurrent neural 
network, then there exists a 'switch' to do so.) 
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Claim 71 

Oh teaches a switch interposed between the processing circuit and the learning 
circuit for selectively enabling use of the learning circuit. (Oh, p1 16:2-1 7; Oh discloses a 
first recurrent neural network (master neural network) which employs the learning phase 
of the neural network or learning circuit of applicant. These values are updated using 
the digital memory. If a user can alter between the first and second recurrent neural 
network, then there exists a 'switch' to do so.) 

Claim 72 

Oh teaches a switch interposed between the processing circuit, the learning 
circuit and the digital memory for selectively enabling the use of either the learning 
circuit or the digital memory by the processing circuit. (Oh, p1 16, 2-17; If there exists 
two phases, a learning phase and a testing phase, then there exists a switch to alter 
between the two phases.) 

Claim 73 

Oh teaches the learning circuit is comprised of a one-dimensional multiplier for 
implementing the update rule. (Oh, abstract; 'One dimensional multiplier' of applicant is 
equivalent to 'vector multiplier' of Oh.) 



Claim 74 



Application/Control Number: 10/773,050 Page 13 

Art Unit: 2129 

Oh teaches the learning circuit is configured to receive an error signal indicative 
of a difference between an output signal and a target output signal. (Oh, p15:9 through 
p16:7; 'Output signal' and 'target output signal' of applicant is equivalent to 'actual 
output' and 'desired output' of Oh.) 

Claim 73 

Oh teaches multiple arrays of synaptic cells, which are interconnected to form a 
multi-layer neural network. (Oh, figure 6.2; Figure 6.2; The 2 dimensional array which 
each node is interconnected with other nodes of Oh is equivalent to 'multi-layer neural 
network' of applicant.) 

Claim 76 

Oh teaches the learning circuit further comprises a second multiplier configured 
to receive the error signal and generates a feedback propagation error signal for a 
previous layer in the multi-layer neural network.(Oh, p1 7:5-6 and equation (2.1 1); 'Error 
signal' of applicant is equivalent to 'w k ' of Oh. 'Previous layer' of applicant is equivalent 
to 'w k " 1 'ofOh.) 

Claim 78 

Oh teaches a digital-to-analog converter interposed between the processing 
circuit and the digital memory in each synaptic cell. (Oh, p1 1 6:2-1 7; Oh discloses a 
digital to analog converter for outputting the value of the synaptic cell.) 
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Claim 79 

Oh teaches wherein each processing circuit in a column of synaptic cells outputs 
a component of a weighted sum. (Oh, p13:4 through p15:8; 'Processing circuit in a 
column' of applicant is equivalent to 'the 'hidden layer' of Oh. 'Outputs a component of 
a weighted sum' of applicant is equivalent to the output of each node in the hidden layer 
is a result of the weighted sum input.) 

Claim 80 

Oh teaches wherein an output of each multiplier in a column of synaptic cells is 
interconnected to form a weighted sum. (Oh, figure 5.5; Oh discloses the output of each 
multiplier (Y r Y 4 ) is fed back as a column to form an interconnection of synaptic cells.) 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 
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Claims 58, 77, 81-88 is rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Oh and Lehmann as set forth above, in view of Newton. 
('Newton's Telecom Dictionary', referred to as Newton) 

Claims 58, 77 

Oh and Lehmann do not teach wherein the digital memory is implemented using 
flip-flops. 

Newton teaches wherein the digital memory is implemented using flip-flops. 
(Newton, p316, C2:43-45; It is common knowledge that a flip flop circuit is used to hold 
digital information.) It would have been obvious to a person having ordinary skill in the 
art at the time of applicant's invention to modify the teachings of Oh and Lehmann by 
disclosing the flip flop circuit as taught by Newton to have the digital memory is 
implemented using flip-flops. 

For the purpose of using industrial standard technology for containing digital 

values. 



Claim 81 

Oh teaches an array of synaptic cells which are interconnected to form a 
feedforward neural network (Oh, p4:1 9 through p5:2), 
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Oh does not teach wherein each synaptic cell includes: a learning electrical 
circuit operable to update a local weight according to an update rule and store the local 
weight in a capacitor. 

Lehmann teaches wherein each synaptic cell includes: a learning electrical circuit 
operable to update a local weight according to an update rule and store the local weight 
in a capacitor. (Lehmann, abstract, p61, C1:36 through C2:21: 'Learning electrical 
circuit operable to update a local weight according to an update rule' of applicant is 
equivalent to 'analog storage (a capacitor) which would accumulate weight changes 
during learning' of Lehmann.) It would have been obvious to a person having ordinary 
skill in the art at the time of applicant's invention to modify the teachings of Oh by using 
a capacitor as taught by Lehman to have wherein each synaptic cell includes: a learning 
electrical circuit operable to update a local weight according to an update rule and store 
the local weight in a capacitor. 

For the purpose of having a smaller footprint on the chip. 

Oh teaches wherein the learning circuit is comprised of a one-dimensional 
multiplier for implementing the update rule (Oh, abstract; 'One dimensional multiplier' of 
applicant is equivalent to 'vector multiplier' of Oh.), and is configured to receive an error 
signal indicative of a difference between an output signal and a target output signal. 
(Oh, p1 5:9 through p1 6:7; 'Output signal' and 'target output signal' of applicant is 
equivalent to 'actual output' and 'desired output' of Oh.) 

Oh does not teach a digital memory operable to store the local weight in a digital 

form. 
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Lehmann teaches a digital memory operable to store the local weight in a digital 
form. (Lehmann, abstract, p61 , C1 :36 through C2:21 : 'A digital memory operable to 
store the local weight in a digital form' of applicant is equivalent to digital weight' of 
Lehmann.) It would have been obvious to a person having ordinary skill in the art at the 
time of applicant's invention to modify the teachings of Oh by using digital memory as 
taught by Lehman to have a digital memory operable to store the local weight in a digital 
form. 

For the purpose of having a stable source of reliable information without 
concerns of capacitor storage accuracy. 

Oh and Lehmann do not teach wherein the digital memory is implemented using 
flip-flops. 

Newton teaches wherein the digital memory is implemented using flip-flops. 
(Newton, p316, C2:43-45; It is common knowledge that a flip flop circuit is used to hold 
digital information.) It would have been obvious to a person having ordinary skill in the 
art at the time of applicant's invention to modify the combined teachings of Oh and 
Lehmann by using flip flops as taught by Newton to have wherein the digital memory is 
implemented using flip-flops. 

For the purpose of using established circuit design for reliable results. 

Oh teaches a processing electrical circuit comprised of a one-dimensional 
multiplier which is operable to process a component of an analog input signal in 
accordance with the local weight stored in the digital memory (Oh, abstract; 'One 
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dimensional multiplier' of applicant is equivalent to 'vector multiplier' of Oh.) a switch 
interposed between the processing circuit, the learning circuit, and the digital memory 
for selectively enabling the use of either the learning circuit or the digital memory by the 
processing circuit(Oh, p1 16, 2-17; If there exists two phases, a learning phase and a 
testing phase, then there exists a switch to alter between the two phases.); a digital-to- 
analog converter interposed between the processing circuit and the digital memory in 
each synaptic cell. (Oh, p1 1 6:2-1 7) 

Claim 82 

Oh teaches multiple arrays of synaptic cells which are interconnected to form a 
multi-layer neural network (Oh, figure 6.2; Figure 6.2; The 2 dimensional array which 
each node is interconnected with other nodes of Oh is equivalent to 'multi-layer neural 
network' of applicant.), wherein the learning circuit further comprises a second multiplier 
configured to receive the error signal and generates a feedback propagation error signal 
for a previous layer in the multi-layer neural network. (Oh, p1 7:5-6 and equation (2.1 1); 
'Error signal' of applicant is equivalent to 'w k ' of Oh. 'Previous layer' of applicant is 
equivalent to 'w k " 1 ' of Oh.) 

Claim 83 

Oh teaches an output of each multiplier in a column of synaptic cells is 
interconnected to form a weighted sum, and each processing circuit in a column of 
synaptic cells outputs a component of the weighted sum. (Oh, figure 5.5; Oh discloses 
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the output of each multiplier (Y1-Y4) is a result of the summation of weights for a given 
row. The output of each multiplier is returned into the interconnected array as a 
column.) 

Claim 84 

Oh teaches wherein each column of synaptic cells in the array of synaptic cells 
represents a neuron in the neural network. (Oh, Figure 6.2; One function of a neuron is 
to use its output for training to convergence. The column(s) of Figure 6.2 illustrate this 
by using the outcome of 'error' circuit for training purposes.) 

Claim 85 

Oh teaches an array of control cells positioned adjacent to the array of synaptic 
cells, where each control cell controls conversion of the local weight from an analog 
form to a digital form for a row of synaptic cells in the array of synaptic cells. (Oh, 
p1 16:2-17; Oh discloses an analog to digital converter to have analog input stored in 
digital form.) 

Claim 86 

Oh teaches wherein the control cells operate synchronously with each other to 
control conversion. (Oh, p12:14 through p13:3; Oh discloses in the McCulloch-Pitts 
model, in which neurons operate in asynchronous manner. Using Oh's model in which 
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the feedforward neural network is interconnected allows synchronously operations to 
occur.) 

Claim 87 

Oh teaches wherein each control cell includes a multiplexer configured to select 
a synaptic cell in a given row of synaptic cells (Oh, figure 6.2; Figure 6.2 illustrates 
numerous multiplexers within the recurrent neural network.) and an analog-to-digital 
converter. (Oh, p1 16:2-17) 

Claim 88 

Oh teaches wherein the analog-to-digital converter for a given row of synaptic 
cells is interconnected to each synaptic cell in the given row of synaptic cells. (Oh, 
p116:2-17) 

Response to Arguments 

5. Applicant's arguments filed on December 19, 2007 for claims 43-48, 51-67, 70- 
88 have been fully considered but are not persuasive. 



6. 



In reference to the Applicant's argument: 
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Applicant argues that Roenle does not teach weights being stored in both a 
capacitor and in digital form. 
Examiner's response: 

Examiner agrees and found another art of reference, Lehmann which discloses 
both the use of a capacitor memory and digital memory forms in the same uses as the 
invention. Office Action stands. 



Conclusion 

7. The prior art of record and not relied upon is considered pertinent to the 
applicant's disclosure. 

-'Analog VLSI implementation of neural systems' Mead 

-'Computer simulation in brain science': Cotterill 

-'A four-quadrant subthreshold mode multiplier for analog neural-network 
applications': Coue, D. 

-'Programmable current-mode neural network for implementation in analogue 
MOS VLSI': TH Borgstrom 

-'A Dedicated Multi-Chip Programmable System for Cellular Neural Networks': M 
Salerno 

-'Programmable-weight building blocks for analog VLSI neural network 
processors': RC Chang 
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8. Claims 43-48, 51 -67, 70-88are rejected. 



Correspondence Information 

9. Any inquiry concerning this information or related to the subject disclosure should 
be directed to the Examiner Peter Coughlan, whose telephone number is (571) 272- 
5990. The Examiner can be reached on Monday through Friday from 7:15 a.m. to 3:45 
p.m. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor David Vincent can be reached at (571) 272-3080. Any response to this 
office action should be mailed to: 

Commissioner of Patents and Trademarks, 

Washington, D. C. 20231; 
Hand delivered to: 

Receptionist, 

Customer Service Window, 
Randolph Building, 
401 Dulany Street, 
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Alexandria, Virginia 22313, 

(located on the first floor of the south side of the Randolph Building); 
or faxed to: 

(571) 272-3150 (for formal communications intended for entry.) 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions 
on access to Private PAIR system, contact the Electronic Business Center (EBC) at 
866-217-9197 (toll free). 
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